Electronic device including pin hole array mask above optical image sensor and related methods

ABSTRACT

An electronic device may include an optical image sensor and a pin hole array mask layer above the optical image sensor. The electronic device may also include a display layer above the pin hole array mask layer that includes spaced apart display pixels, and a transparent cover layer above the display layer defining a finger placement surface capable of receiving a finger adjacent thereto.

TECHNICAL FIELD

The present invention relates to the field of electronics, and, moreparticularly, to the field of optical image sensors.

BACKGROUND

Fingerprint sensing and matching is a reliable and widely used techniquefor personal identification or verification. In particular, a commonapproach to fingerprint identification involves scanning a samplefingerprint or an image thereof and storing the image and/or uniquecharacteristics of the fingerprint image. The characteristics of asample fingerprint may be compared to information for referencefingerprints already in a database to determine proper identification ofa person, such as for verification purposes.

A fingerprint sensor may be particularly advantageous for verificationand/or authentication in an electronic device, and more particularly, aportable device, for example. Such a fingerprint sensor may be carriedby the housing of a portable electronic device, for example, and may besized to sense a fingerprint from a single-finger.

Where a fingerprint sensor is integrated into an electronic device orhost device, for example, as noted above, it may be desirable to morequickly perform authentication, particularly while performing anothertask or an application on the electronic device. In other words, in someinstances it may be undesirable to have a user perform an authenticationin a separate authentication step, for example switching between tasksto perform the authentication.

SUMMARY

An electronic device may include an optical image sensor and a pin holearray mask layer above the optical image sensor. The electronic devicemay also include a display layer above the pin hole array mask layer.The display layer includes a plurality of spaced apart display pixels.The electronic device may further include a transparent cover layerabove the display layer defining a finger placement surface capable ofreceiving a finger adjacent thereto.

The electronic device may also include a light source capable ofdirecting light into the finger when adjacent the transparent coverlayer, for example. The optical image sensor, pin hole array mask layer,and finger placement surface may be configured to define overlappingareas at the finger placement surface, and spaced apart areas at theoptical image sensor.

The pin hole array mask layer may have a plurality of openings eachhaving a size in a range of 5-40 microns. The plurality of openings maybe spaced from one another by a distance in a range of 1-3 millimeters,for example.

The pin hole array mask layer may be spaced from the optical imagesensor by a distance in a range of 100-300 microns. The pin hole arraymask layer may be spaced from the finger placement surface by a distancein a range of 1500-2000 microns, for example.

The pin hole array mask layer may include chromium, for example. Theelectronic device may also include a flexible circuit substrate carryingthe optical image sensor. The optical image sensor may include anintegrated circuit (IC) substrate and image sensing circuitry carried bythe IC substrate.

The electronic device may also include an optically transparent bodybetween the optical image sensor and the pin hole array mask layer. Theelectronic device may further include an optically clear adhesive layerabove the optical image sensor, for example.

The optical image sensor may be capable of performing at least one of anauthentication function, a spoof detection function, a navigationfunction, and a vital sign measurement function, for example. Theoptical image sensor may be capable of performing an authenticationfunction based upon a fingerprint from the finger.

The display layer may include a touch display layer, for example. Thepin hole array mask layer may have a plurality of spaced apart openingstherein, and the pin hole array mask layer may include a plurality oflenses within the plurality of openings.

A method aspect is directed to a method of making an electronic device.The method includes positioning a pin hole array mask layer above anoptical image sensor and positioning a display layer above the pin holearray mask layer. The display layer includes a plurality of spaced apartdisplay pixels. The method also includes positioning a transparent coverlayer above the display layer defining a finger placement surfacecapable of receiving a finger adjacent thereto.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of an electronic device according to anembodiment.

FIG. 2 is a schematic block diagram of an electronic device of FIG. 1.

FIG. 3 is a schematic cross-sectional view of a portion of theelectronic device of FIG. 1.

FIGS. 4a and 4b are graphs comparing light and integration line numbersrelative to a frame according to rolling shutter techniques.

FIG. 5 is a graph comparing light to integration line number relative toa frame according to a global shutter mode.

FIGS. 6a and 6b are graphs of estimated image and object planeresolution respectively for the optical image sensor of the electronicdevice of FIG. 1.

FIGS. 7a and 7b are graphs of estimated imaging resolution in terms ofpoint-spread function shape for the optical image sensor of theelectronic device of FIG. 1.

FIGS. 8a-8g are simulated images illustrating resolution of the opticalimage sensor for a given diameter of an opening in the pin hole arraymask layer of the electronic device of FIG. 2.

FIG. 9 is a schematic cross-sectional view of a prototype electronicdevice for generating images according to the optical image sensingprinciples of the electronic device of FIG. 1.

FIGS. 10a-10h are images captured using the prototype electronic deviceof FIG. 9 illustrating image resolution.

FIGS. 11a-11h are simulated images using the prototype electronic deviceof FIG. 9 illustrating image resolution.

FIGS. 12a-12c are examples of separate sub-images of overlapping objectareas from the prototype electronic device of FIG. 9.

FIGS. 13a-13b are captured images illustrating restoral of a singleimage from overlapping sub-images from the prototype electronic deviceof FIG. 9.

FIG. 14 is a captured image at a relatively low angle using a frontilluminated prototype device.

FIGS. 15a-15d are captured images using different colored light with thefront illuminated prototype device.

FIGS. 16a-16c are captured images at a relatively high angle using thefront illuminated prototype device.

FIGS. 17a-17c are captured images at a relatively high angle using thefront illuminated prototype device.

FIGS. 18a-18c are captured images at a relatively high angle using thefront illuminated prototype device.

FIG. 19 is a schematic cross-sectional view of a portion of anelectronic device according to another embodiment of the presentinvention.

FIG. 20 is a schematic cross-sectional view of a portion of anelectronic device according to another embodiment of the presentinvention.

FIG. 21 is an enlarged schematic cross-sectional view of a portion of anelectronic device according to another embodiment of the presentinvention.

FIG. 22 is an enlarged schematic cross-sectional view of a portion of anelectronic device according to another embodiment of the presentinvention.

DETAILED DESCRIPTION

The present invention will now be described more fully hereinafter withreference to the accompanying drawings, in which preferred embodimentsof the invention are shown. This invention may, however, be embodied inmany different forms and should not be construed as limited to theembodiments set forth herein. Rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art. Likenumbers refer to like elements throughout, and prime notation is used torefer to like elements in different embodiments.

Referring initially to FIGS. 1 and 2, an electronic device 20illustratively includes a housing, for example, a portable housing 21,and a processor 22 carried by the portable housing. The electronicdevice 20 is illustratively a mobile wireless communications device, forexample, a cellular telephone. The electronic device 20 may be anothertype of electronic device, for example, a tablet computer, laptopcomputer, wearable computer, etc.

A wireless transceiver 25 is also carried within the housing 21 andcoupled to the processor 22. The wireless transceiver 25 cooperates withthe processor 22 to perform at least one wireless communicationsfunction, for example, for voice and/or data. In some embodiments, theelectronic device 20 may not include a wireless transceiver 25 or otherwireless communications circuitry.

A display 23 is also carried by the portable housing 21 and is coupledto the processor 22. The display 23 may be a light emitting diode (LED)display, for example, and may have additional circuitry to provide touchdisplay features, as will be appreciated by those skilled in the art.Further details of the display 23 are described below.

A memory 26 is also coupled to the processor 22. The memory 26 is forstoring finger matching biometric template data, for example. The memory26 may store other or additional types of data.

As will be appreciated by those skilled in the art, if the display 23 isin the form of a touch display, the touch display acts as both an inputdevice and a display. As such, the display 23 would cooperate with theprocessor 22 to perform one or more device functions in response toinput. For example, a device function may include a powering on or offof the electronic device 20, initiating communication via the wirelesstransceiver 25, and/or performing a menu function based upon input tothe touch display.

More particularly, with respect to a menu function, the processor 22 maychange the display 23 to show a menu of available applications basedupon pressing or input to the touch display. Of course, other devicefunctions may be performed based upon input to the touch display 23.Other or additional finger-operated user input devices may be carried bythe portable housing 21, for example, a pushbutton switch 24, which mayalternatively or additionally be used for device functions as will beappreciated by those skilled in the art.

Referring now additionally to FIG. 3, an optical image sensor 31 forsensing a biometric of a user, such as, for example, an image of thefingerprint patterns of the user's finger 40, is carried by the housing21 under the display 23. More particularly, the optical image sensor 31includes an integrated circuit (IC) substrate, and image sensingcircuitry carried by the IC substrate. The optical image sensor 31 maybe coupled to a circuit substrate, for example, a flexible substrate 34by way of a grid array having ball grid array (BGA) contacts 35 or othercoupling technique. The optical image sensor 31 may be aback-illuminated sensor or backside illumination (BSI) image sensor aswill be appreciated by those skilled in the art.

The electronic device 20 optionally includes a light source 41. Thelight source 41 directs light into the user's finger 40, and may directlight for the optical image sensor 31. The light source 41 may be one ormore light emitting diodes (LEDs) and/or may be part of the displaylayer 36. In other words, the display pixels 38 may be the light sourceor there may be a separate or additional light source. For example,different LEDs of the display may allow dynamic changing of and/or moreflexibility with respect to the wavelengths of the light and the angleof illumination. A visible light source or invisible light source (e.g.,infrared (IR) or ultraviolet (UV)), and/or another type of light sourcemay be used, or a combination of light sources may be used. However, IRlight may penetrate deeper within a user's finger 40, compared to othercolors of light, for example, blue-colored light. It may be desirablethat the light source 41 be synchronized with the optical image sensor31, and more particularly, signal acquisition of the optical imagesensor. For example, the light source 41 may cooperate with the opticalimage sensor 31 so that the optical image sensor operates in one or bothof a rolling shutter mode and a global shutter mode, as will beappreciated by those skilled in the art. The global shutter mode mayimprove tolerance to background light or interference and reduce powerconsumption as will be appreciated by those skilled in the art.Additionally, the optical image sensor 31 may cooperate with a filter,for example, a narrow band spectral filter, that may correspond to thespectra of the light source 41. The filter may reduce background effectson finger recognition or increase tolerance to the background. Thefilter may be an optical filter, for example.

Further details of the rolling shutter and global shutter modes will nowbe described. A typical optical image sensor generally operates in arolling shutter mode. In this mode, the integration time starts and endsat different times for each sensor line. Such operation may beinefficient when combined with active illumination, as it generallyrequires illumination to be turned in one of two regimes.

Referring now to the graph in FIG. 4a , in a first regime, illumination,i.e., the light source 41, is on from the start of the first lineintegration to the end of the last line integration. This regime has twodisadvantages: 1) the integration time is shorter than the illuminationon time, causing illumination power inefficiency; and 2) if theillumination switch is between the consecutive frames, such as to changethe angle or wavelength, the next frame start is delayed until the firstframe ends, adding a wait time no shorter than the readout time, causinga time inefficiency.

Referring now to the graph in FIG. 4b , in a second regime, theillumination is on from the start of the last line integration to theend of the first line integration. This regime has two disadvantages: 1)the integration time is longer than the illumination on time, causingbackground light interference inefficiency; and 2) the illumination dutycycle is relatively short causing a high peak power operation.

Referring now to the graph in FIG. 5, it may be thus desirable tooperate the optical image sensor 31 in a global shutter mode. In thismode the integration time starts and ends at the same time for allsensor lines. Such operation has three advantages: 1) The illuminationon time is equal to the integration time causing efficient use of theillumination power; 2) there is typically no need to have dead timebetween the frames in case of illumination switching such as to changeangle or wavelength; and 3) the illumination duty cycle is maximum,relaxing the need for high peak power operation.

An optically clear adhesive layer (OCA) 42 is above the optical imagesensor 31, and more particularly, carried by an upper surface of theoptical image sensor. An optically transparent body or support member 43spaces the OCA layer 42 from a pin hole array mask layer 50. In otherwords, the support member 43 is between the OCA layer 42 and the pinhole array mask layer 50.

The pin hole array mask layer 50 is above the optical image sensor 31,for example, spaced from the optical image sensor by a distance in arange of 100-300 microns. More particularly, the pin hole array masklayer 50 is illustratively carried on a top surface of the supportmember 43. The pin hole array mask layer 50 is an opaque mask and has aplurality of openings 51 or pin holes therein to permit the passage oflight therethrough. The openings 51 may be uniformly spaced or spaced ina honeycomb pattern, for example. The pitch of spacing of the openings51 may be, for example, in a range of 1-3 mm and more particularly,about 1.5 mm. As will be appreciated by those skilled in the art, thespacing between the openings 51 or pitch affects image resolution.Additionally, each opening 51 may have a size in the range of 5-40microns, for example. Of course, the size of each opening 51 or pin holeaffects the sensed images from the optical image sensor 31, as will bedescribed in further detail below. The pin hole array mask layer 50 isopaque, and thus does not permit light to pass through. The pin holearray mask layer 50 may include chromium, for example, a layer ofchromium, to provide the opacity. Of course, other materials, whether ina layer or not, may be used to provide opacity.

A display layer 36, which is part of the display 23, is above the pinhole array mask layer 50. The display layer 36 illustratively includesan array of display pixels 38 and/or micro-lenses for displaying images,as will be appreciated by those skilled in the art. In particular, thedisplay layer 36 may be part of a light-emitting diode (LED) display.The LEDs or display pixels 38 may be spaced apart to allow light to passthrough, and may be aligned with the openings 51 or pin holes.

A display encapsulation layer 44 is over the display layer 36. Anotheroptically clear adhesive layer 45 is over the display encapsulationlayer 44. A transparent cover layer 46, for example, that includes onyx,is above the display layer 36 and defines a finger placement surfacethat is capable of receiving the user's finger adjacent thereto. Moreparticularly, the transparent cover layer 46 is carried by the opticallyclear adhesive layer 45, and an upper surface of the transparent coverlayer 46 defines the finger placement surface 47 for receiving theuser's finger 40. The finger placement surface 47 may be spaced from thepin hole array mask layer 50 by a distance in a range of 1.5 mm-2 mm(i.e., 1500-2000 microns), for example. Of course, the finger placementsurface 47 may be spaced from the pin hole array mask layer 50 byanother distance, for example, based upon desired image characteristics.

In an exemplary electronic device 20 that includes the optical imagesensor 31, the height of the layers may be as follows: the flexiblesubstrate 34 may be about 0.15 mm thick, the optical image sensor 31 maybe about 0.1 mm, the optically clear adhesive layer 42 may be about 0.05mm, the support member 43 may be about 0.2 mm, the display encapsulationlayer 44 may be about 0.1 mm, the second optically clear adhesive layer45 may be about 0.15 mm, and the transparent cover layer 46 about 1.5mm. Of course, the spacing between and size of each layer may bedifferent, but as will be described below it may be desirable that thespacing between the optical image sensor 31 and the pin hole array masklayer 50 be relatively small.

The relative spacing and geometry of the optical image sensor 31, thepin hole array mask array layer 50, and the finger placement surface 47define overlapping areas at the finger placement surface, and spacedapart areas at the optical image sensor. Accordingly, the spacingbetween the pin hole array mask layer 50 and the optical image sensor 31determines an amount of sensed image overlap, i.e., at the fingerplacement surface 47. A larger spacing corresponds to a larger amount ofimage overlap which may be undesirable for processing. In other words,the more overlap, the more computationally intense image constructionmay be. In contrast, a smaller distance between the optical image sensor31 and the pin hole array layer 50 may result in no significant overlap,and thus, images may be more easily reconstructed.

The optical image sensor 31, and more particularly, the image sensingcircuitry senses a user's finger 40 or an object placed adjacent thefinger placement surface 47, and based thereon, may perform one or morebiometric functions, for example, user authentication (a matchingoperation), a biometric enrollment function, and/or a spoof detectionfunction. Moreover, when the display 23 is in the form of a touchdisplay, when the user contacts the touch display, for example, during anavigation function or other touch display input, data from the user'sfinger 40 is sensed or acquired by the optical image sensor 31, forexample, for finger matching and/or spoof detection, as will beappreciated by those skilled in the art.

Operation of the electronic device 20 as it pertains to finger biometricsensing using the optical image sensor 31 will now be described. Lightfrom the light source 41 and/or display pixels 38 is scattered basedupon an object, for example, the user's finger 40, adjacent the fingerplacement surface 47 or on the transparent cover layer 46. The scatteredlight is captured by the optical image sensor 31 through the pin holesand/or micro-lenses in the display layer 36 and the openings 51 or pinholes in the pin hole array mask layer 50.

Advantageously, the display layer 36 is a multi-spectral andmulti-shadow illuminator and generally not affected by ambient light.Moreover, in some embodiments, the display layer 36 may be used forspoof detection, for example, impedance based spoof detection and/orother light-based or electric field-based detection techniques, as willbe appreciated by those skilled in the art.

Even still further, the die of the optical image sensor 31 has arelatively large amount of non-utilized areas, which can be allocatedfor other processing, for example, finger biometric or fingerprintprocessing and/or spoof detection, e.g. a spectrometer.

Using a pin hole array mask layer 50 as part of an imaging techniqueproduces separate images of overlapping object areas. Shading andmagnification of the image may be adjusted by adjusting differentparameters with respect to size and distance from object to the pin holearray layer 50 and to the optical image sensor 31. For example, amagnification of 0.114 can be achieved based upon the height and theaverage refractive index ratio. Opening or pin hole image shading isgiven by a cos⁴ function. Shading allows for separation of the sensedimages, even though there are overlapping areas. Moreover, the shadingdetermines the effective size of the object area images by a singleopening 51 or pin hole.

With respect to image overlap, using a signal level in the range of64%-100%, a field-of-view angle of ±26.5° may be obtained. When usedwith an opening 51 or pin hole size of 200 microns, an object area sizeof 1750 microns, and a spacing or pitch of the openings of 1500 microns,the object may be full covered by the imaged areas. By using a signallevel in the range of 20%-100%, a field-of-view angle of ±48° may beobtained. When used with an opening 51 or pin hole size of 200 microns,an object area size of 1750 microns, and a pin hole spacing or pitch ofthe openings of 1500 microns, each object area is sensed or imagedmultiple times from different angles in the same capture. The overlapinformation may be used to improve resolution and signal-to-noise ratio(SNR), and/or extract 3D information, for example.

With respect to resolution, the use of the pin hole array layer 50allows image resolution of about 15 microns. Thus, a relatively widerange of pixel sizes may be used. For example, an object planeresolution of about 120 microns may be achieved.

More particularly, the pin hole optical system resolution may bedetermined based upon a pin hole imaging point spread function (PSF)that is a convolution of geometrical and diffraction PSF. Both areaxially symmetric 2D functions. The geometrical PSF quantifies blurringdue to the finite size of each opening or pin hole. The geometrical PSFis given by the pin hole circle projections onto the optical imagesensor 31 (for image space resolution) or onto the object (for objectspace resolution). The diffraction PSF quantifies the additionalblurring due to the light diffraction off small openings, for example,for a circular aperture, it is given by the first Bessel function.

Referring to FIGS. 6a and 6b , the graphs 60 a and 60 b are graphs ofopening 51 or pin hole size based upon an object-to-opening distance of1750 microns and an optical image sensor-to-opening distance of 200microns. The graphs 60 a, 60 b plot the diameter of each opening 51 inmicrons against the image plane resolution (FIG. 6a ) and the objectplane resolution (FIG. 6b ) in microns, respectively. Lines 61 a, 61 bcorrespond to light having a wavelength of 380 nm, lines 62 a, 62 bcorrespond to light having a wavelength of 460 nm, lines 63 a, 63 bcorrespond to light having a wavelength of 525 nm, lines 64 a, 64 bcorrespond to light having a wavelength of 630 nm, lines 65 a, 65 bcorrespond to light having a wavelength of 850 nm, and lines 66 a, 66 bcorrespond to light having a wavelength of 940 nm. The size of theopenings 51 that may be particularly well suited for visible light is 9microns.

Additionally, the lines' PSF width rise at relatively large pin holes oropenings 51 is the geometric resolution-dominant regime. The fast PSFwidth rise at relatively smaller openings 51 is the diffraction-dominantregime. The two effects combined produce what may be considered anoptimum pin hole size for the best resolution. It may be desirable thatthe selection of the openings 51 size be somewhat above the optimumdetermined resolution, for example, to trade-off resolution forsignal-to-noise ratio (SNR).

Referring now to the graphs 70 a, 70 b in FIGS. 7a and 7b , pin holeimaging resolution is illustrated. The graph 70 a in FIG. 7a correspondsto an opening 51 or pin hole size of 9 microns, an object-to-openingdistance of 1750 microns, and an image sensor-to-opening distance of 200microns, while the graph 70 b in FIG. 7b corresponds to an opening orpin hole size of 15 microns, an object-to-opening distance of 1750microns and an image sensor-to-opening distance of 200 microns. Lines 71a, 71 b correspond to light having a wavelength of 460 nm, lines 72 a,72 b correspond to light having a wavelength of 525 nm, lines 73 a, 73 bcorrespond to light having a wavelength of 630 nm, and lines 74 a, 74 bcorrespond to light having a wavelength of 850 nm. Illustratively, foran opening 51 or pin hole diameter of 9 microns, the object planeresolution (1/e) is 105 microns, while for an opening 51 or pin holediameter of 15 microns, the object plane resolution (1/e) is 155microns. The graph 70 a corresponds to a relatively small pin hole size,a diffraction regime, has a bell shape, and thus, a relativelysignificant wavelength dependence. The graph 70 b corresponds to arelatively large pin hole size, a mostly geometrical regime, has asquare shape, and thus negligible wavelength dependence.

It is also desirable to account for pixel blurring. The pixel PSF is aconvolution of pixelization and crosstalk PSF. The pixelization PSF isdue to the finite size of the pixel, and it can be modeled by a2D-square sinc function or by integrating a super-sampled image.

The crosstalk PSF is the pixel property that is measured, for example,by way of angle and wavelength. The crosstalk PSF depends on theincoming angle, and more particularly, on pixel position with respect tothe image center. The crosstalk PSF typically is of the order of onepixel in size, but can have a long-range tail, especially for nearinfrared (NIR) light, for example. Pixel blurring, however, is notgenerally expected to be relatively significant compared to opticalblurring since the pixel size is significantly smaller than the size ofthe openings 51 or pin holes.

Referring now to FIGS. 8a -8 g, simulated images illustrating exemplaryresolutions are illustrated. The images are for green light, an opening51 diameter of 15 microns, and a resolution of 155 microns. FIG. 8Aillustrates 4 lines per millimeter with a line width of 125 microns.FIG. 8B illustrates 5 lines per millimeter with a line width of 100microns. FIG. 8C illustrates 6 lines per millimeter with a line width of83 microns. FIG. 8D illustrates 7 lines per millimeter with a line widthof 71 microns. FIG. 8E illustrates 8 lines per millimeter with a linewidth of 63 microns. FIG. 8F illustrates 9 lines per millimeter with aline width of 56 microns, and FIG. 8G illustrates 10 lines permillimeter with a line width of 50 microns. A 1/e resolution of 155microns advantageously allows for resolving of up to about 7 lines permillimeter, which may depend on a contrast degradation limit, forexample.

With respect to shading, shading includes both optical shading and pixelshading. Optical shading can be approximated by the “cosine-4^(th)”geometrical factor. Light is received at the optical image sensor 31 atangles that depends on the refractive index ratio of the pin hole plane.The pixel shading is measured and is expected to be no more than anextra cosine factor in addition to the geometrical effect.

With respect to signal-to-noise ratio (SNR) and integration time, thesize of each opening 51 or pin hole drives the resolution-SNR trade off.The signal level is based upon pin hole plane irradiance, opening 51 orpin hole size, pixel sensitivity, integration time, and shading. Thenoise level for a given optical image sensor may be a function of thesignal with constant parameters including pixel properties, such as forexample, read noise, photo response non-uniformity (PRNU), and fixedpattern noise (FPN).

For example, for a resolution-optimal opening diameter of 9 microns, theF-number is 22.2. For an opening diameter of 15 microns, with a sensordistance of 200 microns, the F-number is 13.3 (a resolution loss ofabout 1.5×, and an integration time reduction for the same SNR of about2.8×). As will be appreciated by those skilled in the art, the imagecenter signal is given by:

Signal=luminance[cd/m²]*π*reflectivity*transmissivity/(4F ²)*sensitivity[e/lx−s]*tau[s]

For a typical display luminance of about 520 cd/m², reflectivity ofabout 70%, F/13.3, pixel pitch 6 microns, integration time of 100 ms,the resultant signal may be about 140e, with an SNR of about 11. ThisSNR may be considered relatively low, and thus it may be desirable forthe image modulation to be about 10% for a workable contrast. A largereffective pixel pitch, i.e., spacing between pixels, for example viabinning, may be considered for an SNR increase or an integration timedecrease.

With respect to image distortion, image distortion may result based upona fisheye or inverse-fisheye effect. The image distortion may be due tothe difference in the refractive index between the object interfacemedia and the optical image sensor 31, for example, and is modeled bythe sine ratio refraction function. Pin hole imaging itself does notintroduce significant distortion, thus maintaining the angle tangentsrelatively constant. Distortion may be reduced by using materials havinga closer match, for example nearly the same refractive index. Distortionmay be corrected by image processing before stitching togetherindividual images, as will be appreciated by those skilled in the art.

Referring now to FIG. 9, a prototype electronic device 200 was used togenerate images according to the principles described above. Moreparticularly, a chromium mask 201 having a thickness of about 12 micronscarried by a back glass 202 having a refractive index of 1.5 and athickness of 1500 microns was used to simulate an object to be sensed.The chromium mask 201 simulating the object was a Thorlabs R2L2S1Ppositive resolution target having a 2-inch by 2-inch size, a soda limeglass substrate (back glass) and a chromium pattern.

A diffused light source 203 was positioned above the chromium mask 201.The diffused light source 203 included multiple blue light-emittingdiodes (LEDs) uniformly illuminating a diffuser over 2 inches. Thecentral wavelength was about 450 nm. The light source 203 was limited toblue LEDs because of a residual transmission of chromium masks at higherwavelengths that caused reduced contrast.

The chromium mask 201 was spaced from a pin hole array mask layer 204 byabout 1500 microns as the pin hole array mask layer was also carried bya back glass 205 having a refractive index of about 1.5 and a thicknessof about 1500 microns. The pin hole array mask layer 204 had a thicknessof about 12 microns, and the diameter of the single opening 206 in thepin hole array mask layer was 12 microns.

An optical image sensor 207 was below the pin hole array mask layer 204and spaced therefrom by about 750 microns with an associated refractiveindex of about 1.3. The 750 micron spacing included a 150 micron air gap208, a cover glass layer 209 with a thickness of 300 microns and arefractive index of 1.5, and a second air gap 210 having a thickness of300 microns. The predicted object plane resolution was 38 microns(PSF−1/e diameter; equivalent to a minimum resolved line-pair width).

Referring additionally to the images in FIGS. 10A-10H and FIGS. 11A-11H,the prototype was used to generate images, which were compared tosimulated images, respectively. FIGS. 10A-10H correspond to captured orgenerated images for 18, 16, 14, 12.5, 11, 10, 9, and 8 lines permillimeter, respectively. FIGS. 11A-11H correspond to the simulatedimages for 18, 16, 14, 12.5, 11, 10, 9, and 8 lines per millimeter,respectively. It should be noted that 18 lines per millimeter is stillresolved, but as illustrated, the contrast is relatively low (line widthof 28 microns). Referring particularly to FIGS. 10E-10H, the visibledistortion is due to the “inverse fisheye” effect to the refractionindex step on the pin hole or opening 51.

Referring now to FIGS. 12A-12C, exemplary images of separate sub-imagesof overlapping object areas are illustrated. The captured images inthese figures were taken from a 5×5 pin hole array layer having a 12micron diameter with 1000 micron spacing. FIGS. 13A and 13B illustraterestoral of a single image from overlapping sub-images, for example,those illustrated in FIGS. 12A-12C.

Further tests were performed using a front illuminated prototype devicethat show that with respect to finger ridge imaging, contrast generallystrongly depends on angle and wavelength. More particularly, withrespect to a front illuminated prototype device the light source waspositioned laterally adjacent the image sensor and was laterallyadjusted for different angles of light. The same chromium mask asdescribed with the prototype above was used to simulate an object to besensed. Referring now to the image in FIG. 14, the ridge image contrastat 550 nm is relatively low when the light source angle is close tonormal, for example.

Referring now to FIGS. 15A-15D, the contrast with blue light (450 nm,FIG. 15A) or green light (550 nm, FIG. 15B) is illustratively betterthan with red light (650 nm, FIG. 15C) or infrared light (940 nm, FIG.15D). FIGS. 16A-16C are captured images at a relatively high angle at550 nm, 850 nm, and 940 nm, respectively. FIGS. 17A-17C and FIGS.18A-18C are additional captured images at the relatively high angle at550 nm, 850 nm, and 940 nm, respectively. Illustratively, the contrastis significantly improved at high angles, but still lower at infraredwavelengths. The ridge density is about 3 lines per millimeter.

While the electronic device 20 has been described herein as being in theform of a mobile wireless communications device, it should be understoodby those skilled in the art that the electronic device may be in theform of a standalone optical image sensing device (i.e., a fingerbiometric sensing or fingerprint sensing device).

Moreover, while the optical image sensor 31 has been described primarilyas being used for biometric authentication, it is understood that theoptical image sensor, and more particularly, the image sensingcircuitry, is capable of performing any or all of a biometricauthentication function, spoof detection function, and a vital signmeasurement function. In particular, the sensed 3D geometry of shadowingusing the pin hole array mask layer 50, the multi-spectral nature of theimaging, and/or other characteristics of live fingers may be used forbiometric authentication, for example. The optical image sensor 31 mayalso be capable of performing sensing other biometric features, such as,for example, heart or pulse rate (which may be used for determining ablood pressure), and/or pulse or blood oximetry, and may be based uponthe ability of the sense images at different wavelengths. As will beappreciated by those skilled in the art, for detecting a heart rate, acombination of green and IR light may be used, and for detecting a bloodoxygen level, a combination of red and IR light may be used.

Still further, the optical image sensor 31 may be used in combinationwith the openings 51 to operate in an ambient light sensing mode, whichmay be relatively desirable in wearable electronic devices, for example.More particularly, by using, for example, the entire pin hole array masklayer 50 and the entire pixel array of the optical image sensor 31, arelatively high angle of light acceptance may result, which is generallydesirable for ambient light sensing operations.

Further details of the operation of the ambient light sensing mode willnow be described. All pixels may be combined into a single output, andread out with extremely low power consumption readout circuitry. Theoptical image sensor 31 in combination with the pin hole array masklayer 50 may then integrate light in very wide field of view (FOV), forexample, up to 180 degrees. A typical camera, for example, senses lightin a relatively narrow FOV, typically between 60 and 70 degrees, whichmay be too small for operation in an ambient light sensing mode. Byhaving a very large, for example, up to 180 degree FOV for the pin holearray mask layer 50 in combination with the optical image sensor 31 mayprovide a relatively large advantage over a typical camera, for example.

The use of the pin hole array mask layer 50, or even pinhole imagingtechniques, provides wide-angle light sensing since the pinholes oropenings 51 are located relatively close to the optical image sensor 31.The effective focal length is thus significantly lower than the size ofthe optical image sensor 31. When the pixels are combined in a singleoutput, it would be sensitive to nearly all the light entering theopenings 51. This allows a relatively low-power ambient light sensingmode that would have a stability advantage over typical sensors due tothe reduced orientation dependence, for example.

Referring now to FIG. 19, a portion of an electronic device 20′illustrating an exemplary integration design. A substrate 34′ is spacedfrom a base 39′. Passive components 49′ are carried by a lower surfaceof the substrate 34′. The optical image sensor 31′ is carried by anupper surface of the substrate 34′. Bond wires 57′ couple the opticalimage sensing circuitry to circuitry carried by the substrate 34′. Anenclosure 58′ extends upwardly from the substrate 34′ around the opticalimage sensor 31′. A transparent glass layer 59′ is carried by theenclosure, and has a thickness, for example, of 0.5 mm. A pin hole arraymask layer 50′ is carried by a lower surface of the transparent layer59′, for example, a glass layer. The enclosure 58′ spaces the pin holearray mask layer 50′ and the transparent glass layer 59′ from theoptical image sensor 31′, for example, by a distance of 150 micronsdefining an air gap therebetween. A light absorptive adhesive 92′, forexample, epoxy, may secure the transparent glass layer 59′ and the pinhole array mask layer 50′ to the enclosure 57′.

Referring now to FIG. 20, the components or elements illustrated in FIG.19 are integrated into an exemplary electronic device 20′. A printedcircuit board (PCB) 81′ couples the substrate 34′, and moreparticularly, the lower surface of the substrate adjacent the passivecomponents 49′. A display layer 36′ including spaced apart displaypixels 38′ is carried by the upper surface of the substrate 34′laterally adjacent, or around, the enclosure 58′. The display layer 36′may be coupled to display control circuitry 82′ carried off thesubstrate 34′. A transparent cover layer 46′ is over the transparentlayer 59′. The transparent cover layer 46′ may be secured to thetransparent layer 59′ with an adhesive, for example. The transparentcover layer 46′ may be glass or onyx, for example, or may be anothermaterial.

A method aspect is directed to a method of making an electronic device20. The method includes positioning a pin hole array mask layer 50 abovean optical image sensor 31, and positioning a display layer 36 above thepin hole array mask layer. The display layer 36 includes spaced apartdisplay pixels 38. The method also includes positioning a transparentcover layer 46 above the display layer 36 defining a finger placementsurface 47 capable of receiving a user's finger 40 adjacent thereto.

Another method aspect is directed to a method of sensing an opticalimage. The method includes using an optical image sensor 31 to senselight reflected from a user's finger 40 adjacent a finger placementsurface 47 defined by a transparent cover layer 46, through thetransparent cover layer, through a pin hole array mask layer 50 abovethe optical image sensor, and through a display layer 36 above the pinhole array mask layer, wherein the display layer includes spaced apartdisplay pixels 38.

Referring now to FIG. 21, in another embodiment, the pin hole array masklayer 50″ including the openings 51″ may not be between the opticalimage sensor 31″ and the display layer 36″, but instead, carried by orintegrated with the display layer. Illustratively, the display layer 36″includes an array of spaced apart display pixels 38″ and/or micro-lensesfor displaying images and which are spaced apart to allow light to passthrough. The spacing between the display pixels 38″, which allows thelight to pass through, defines the openings 51″. This is in contrast toembodiments where the space between the spaced apart display pixels 36may be aligned with the openings 51 or pin holes in the pin hole arraymask layer 50. Method aspects are directed to a method of making arelated electronic device and a method of using or sensing a fingerusing the electronic device.

Referring now to FIG. 22, in yet another embodiment, the pin hole arraymask layer 50″′ includes lenses 91″′ in the openings 51″′. Each of thelenses 91″′ may have a diameter of about 40-100 microns, for example.The lenses 91″′ may advantageously improve image quality and the SNR,which may thus reduce optical power for illumination and total overallpower consumption, which may be particularly, advantageous when usedwith mobile or portable devices. Lenses may alternatively oradditionally be included in the display layer 36″′.

The benefits of biometric data collected by a device as disclosed hereininclude convenient access to device features without the use ofpasswords. In other examples, user biometric data is collected forproviding users with feedback about their health or fitness levels. Thepresent disclosure further contemplates other uses for personalinformation data, including biometric data, that benefit the user ofsuch a device.

Practicing the present invention requires that collecting, transferring,storing, or analyzing user data, including personal information, willcomply with established privacy policies and practices. In particular,such entities should implement and consistently use privacy policies andpractices that are generally recognized as meeting or exceeding industryor governmental requirements for maintaining personal information dataprivate and secure, including the use of data encryption and securitymethods that meets or exceeds industry or government standards. Personalinformation from users should not be shared or sold outside oflegitimate and reasonable uses. Further, such collection should occuronly after receiving the informed consent of the users. Additionally,such entities would take any needed steps for safeguarding and securingaccess to such personal information data and ensuring that others withaccess to the personal information data adhere to their privacy policiesand procedures. Further, such entities can subject themselves toevaluation by third parties to certify their adherence to widelyaccepted privacy policies and practices.

The present disclosure also contemplates the selective blocking ofaccess to, or use of, personal information data, including biometricdata. Hardware and/or software elements disclosed herein can beconfigured to prevent or block access to such personal information data.Optionally allowing users to bypass biometric authentication steps byproviding secure information such as passwords, personal identificationnumbers (PINS), touch gestures, or other authentication methods, aloneor in combination, is well known to those of skill in the art. Users canfurther select to remove, disable, or restrict access to certainhealth-related applications collecting users' personal health or fitnessdata.

Many modifications and other embodiments of the invention will come tothe mind of one skilled in the art having the benefit of the teachingspresented in the foregoing descriptions and the associated drawings.Therefore, it is understood that the invention is not to be limited tothe specific embodiments disclosed, and that modifications andembodiments are intended to be included within the scope of the appendedclaims.

1. An electronic device comprising: an optical image sensor; a pin holearray mask layer above the optical image sensor; a display layer abovethe pin hole array mask layer comprising a plurality of spaced apartdisplay pixels; and a transparent cover layer above the display layerdefining a finger placement surface capable of receiving a fingeradjacent thereto.
 2. The electronic device of claim 1 further comprisinga light source capable of directing light into the finger when adjacentthe transparent cover layer.
 3. The electronic device of claim 2 whereinthe light source comprises at least one of visible light source,infrared light source, and an ultraviolet light source.
 4. Theelectronic device of claim 1 wherein the optical image sensor, pin holearray mask layer, and finger placement surface are configured to defineoverlapping areas at the finger placement surface, and spaced apartareas at the optical image sensor.
 5. The electronic device of claim 1wherein the pin hole array mask layer has a plurality of openings eachhaving a size in a range of 5-40 microns.
 6. The electronic device ofclaim 1 wherein the pin hole array mask layer has a plurality ofopenings spaced from one another by a distance in a range of 1-3millimeters.
 7. The electronic device of claim 1 wherein the pin holearray mask layer is spaced from the optical image sensor by a distancein a range of 100-300 microns.
 8. The electronic device of claim 1wherein the pin hole array mask layer is spaced from the fingerplacement surface by a distance in a range of 1500-2000 microns.
 9. Theelectronic device of claim 1 wherein the pin hole array mask layercomprises chromium.
 10. The electronic device of claim 1 furthercomprising a flexible circuit substrate carrying the optical imagesensor.
 11. The electronic device of claim 1 further comprising anoptically transparent body between the optical image sensor and the pinhole array mask layer.
 12. The electronic device of claim 1 furthercomprising an optically clear adhesive layer above the optical imagesensor.
 13. The electronic device of claim 1 wherein the optical imagesensor is capable of performing at least one of an authenticationfunction, a spoof detection function, a navigation function, and a vitalsign measurement function.
 14. The electronic device of claim 1 whereinthe optical image sensor is capable of performing an ambient lightmeasurement.
 15. The electronic device of claim 1 wherein the opticalimage sensor is capable of performing an authentication function basedupon a fingerprint from the finger.
 16. The electronic device of claim 1wherein the display layer comprises a touch display layer.
 17. Theelectronic device of claim 1 wherein the pin hole array mask layer has aplurality of spaced apart openings therein; and wherein the pin holearray mask layer comprises a plurality of lenses within the plurality ofopenings.
 18. An electronic device comprising: an optical image sensor;a display layer above the optical image sensor and comprising aplurality of spaced apart display pixels and having a plurality of pinholes therebetween; and a transparent cover layer above the displaylayer defining a finger placement surface capable of receiving a fingeradjacent thereto.
 19. The electronic device of claim 18 furthercomprising a light source capable of directing light into the fingerwhen adjacent the transparent cover layer.
 20. The electronic device ofclaim 19 wherein the light source comprises at least one of visiblelight source, infrared light source, and an ultraviolet light source.21. The electronic device of claim 18 wherein the optical image sensor,the plurality of pin holes, and finger placement surface are configuredto define overlapping areas at the finger placement surface, and spacedapart areas at the optical image sensor.
 22. The electronic device ofclaim 18 wherein the plurality of pin holes each have a size in a rangeof 5-40 microns.
 23. The electronic device of claim 18 wherein theplurality of pin holes are spaced from one another by a distance in arange of 1-3 millimeters.
 24. The electronic device of claim 18 furthercomprising an optically clear adhesive layer above the optical imagesensor.
 25. The electronic device of claim 18 wherein the optical imagesensor is capable of performing at least one of an authenticationfunction, a spoof detection function, and a vital sign measurementfunction. 26-33. (canceled)
 34. A method of using an electronic devicecomprising: using a light source to direct light into a finger whenadjacent a transparent cover layer defining finger placement surfacecapable of receiving the finger adjacent thereto; and using an opticalimage sensor to sense reflected light from the finger adjacent thefinger placement surface, the reflected light being sensed through a pinhole array mask layer above an optical image sensor, a display layerabove the pin hole array mask layer, the display layer comprising aplurality of spaced apart display pixels, and the transparent coverlayer above the display layer.
 35. The method of claim 34 wherein thelight source comprises at least one of visible light source, infraredlight source, and an ultraviolet light source.
 36. The method of claim34 wherein the optical image sensor, pin hole array mask layer, andfinger placement surface define overlapping areas at the fingerplacement surface, and spaced apart areas at the optical image sensor.37. The method of claim 34 wherein the pin hole array mask layer has aplurality of openings each having a size in a range of 5-40 microns. 38.The method of claim 34 wherein the pin hole array mask layer has aplurality of openings spaced from one another by a distance in a rangeof 1-3 millimeters.
 39. The method of claim 34 wherein the pin holearray mask layer is spaced from the optical image sensor by a distancein a range of 100-300 microns.
 40. The method of claim 34 wherein thepin hole array mask layer is spaced from the finger placement surface bya distance in a range of 1500-2000 microns.